The anomalous properties of the Hall constant in the normal state of high-$T_c$ superconductors are investigated within the single-band Hubbard model. We argue that the Mori theory is the appropriate formalism to address the Hall constant, since it aims directly at resistivities rather than conductivities. More specifically, the frequency dependent Hall constant decomposes into its infinite frequency limit and a memory function contribution. As a first step, both terms are calculated perturbatively in $U$ and on an infinite dimensional lattice, where $U$ is the correlation strength. If we allow $U$ to be of the order of twice the bare band width, the memory function contribution causes the Hall constant to change sign as a function of doping and to decrease as a function of temperature.